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Resolving  a  -Mixture  of  Strategies  in 
Spatial  Visualization  Tasks 

Abstract 

The  models  of  standard  test  theory,  having  evolved  under  a 
trait-oriented  psychology,  do  not  reflect  the  knowledge  structures 
and  the  problem-solving  strategies  now  seen  as  central  to 
understanding  performance  and  learning.  In  some  applications, 
however,  key  qualitative  distinctions  among  persons  as  to 
structures  and  strategies  can  be  expressed  through  mixtures  of 
test  theory  models,  drawing  upon  substantive  theory  to  delineate 
the  components  of  the  mixture.  This  approach  is  illustrated  with 
response  latencies  to  spatial  visualization  tasks  that  can  be 
solved  by  mental  rotation  or  by  a  nonspatial  rule-based  strategy. 
It  is  assumed  that  a  subject  employs  the  same  strategy  on  all 
tasks,  but  the  possibility  of  extending  the  approach  to  strategy¬ 
switching  is  discussed. 

Key  words:  EM  algorithm,  item-solving  strategies,  mental 

rotation,  mixture  models,  spatial  visualization. 


Introduction 


Recent  research  in  cognitive  and  educational  psychology 
reveals  the  central  role  of  strategies,  mental  models,  and 
knowledge  structures  in  learning  and  problem-solving.  Clancy 
(1986)  describes  the  shift  in  perspective  to  "describing  mental 
processes,  rather  than  quantifying  performance  with  respect  to 
stimulus  variables."  To  this  task,  standard  test  theory,  with  its 
focus  on  overall  level  of  proficiency,  is  ill  suited. 

Consider  as  an  example  a  test  in  which  different  persons 
employ  different  solution  strategies.  By  characterizing  examinees 
simply  in  terms  of  their  propensities  to  make  correct  responses, 
the  models  of  standard  test  theory  (item  response  theory  [IRT]  as 
well  as  classical  test  theory)  cloud  analyses  in  several  ways: 
information  about  subjects'  mental  processes  is  obscured, 
relationships  between  tasks'  features  and  difficulties  are 
confounded  with  strategy  choice,  and  comparisons  of  subjects  in 
terms  of  overall  proficiencies  are  equivocal. 

Mislevy  and  Verhelst's  (in  press)  mixture  model  approach  to 
test  theory  can  be  employed  to  handle  certain  distinctions  of  this 
type,  as  when  different  subjects  employ  different  solution 
strategies.  The  following  assumptions  are  made: 

1.  Potential  strategies  can  be  delineated  a  priori. 

2.  An  examinee  uses  the  same  strategy  on  all  tasks. 
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3.  An  examinee's  responses  are  observed  directly  but  strategy 
choice  is  not. 

4.  The  responses  of  examinees  following  a  given  strategy  conform 
to  a  response  model  of  a  known  form,  possibly  characterized 
by  unknown  parameters. 

5.  For  each  strategy,  psychological  or  substantive  theory 
delineates  relationships  between  the  observable  features  and 
the  difficulties  of  tasks. 

The  present  paper  illustrates  this  approach  by  modelling 
response  latencies  for  spatial  rotation  tasks  that  can  be  solved 
by  either  a  rotational  or  a  rule -based  strategy.  Simplified 
versions  of  processing  models  from  the  visualization  literature 
are  the  basis  of  inference  about  strategy  usage.  An  empirical 
Bayesian  approach  provides  maximum  likelihood  estimates  of  the 
"structural"  parameters  of  the  problem  and,  for  each  subject, 
posterior  probabilities  of  membership  in  each  strategy  class  and 
conditional  estimates  of  proficiency  under  each. 

The  Data 

The  data  were  gathered  with  a  computer  administered  test  of 
what  are  typically  called  "mental  rotation  tasks."  This  area  was 
selected  by  virtue  of  its  long  history  of  research  in  both 
psychometrics  (e,g.,  Michael,  Guilford,  Fruchter,  &  Zimmerman, 
1957;  Thurstone,  1938)  and  cognitive  psychology  (e.g.,  Just  & 
Carpenter,  1985;  Lohman,  Pellegrino,  Alderton,  &  Regian,  1987). 

Of  particular  interest  is  the  finding  that  tasks  of  some  tests 
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designed  to  measure  spatial  visualization  abilities  can  be  solved 
with  nonspatial,  analytic  strategies  (French,  1965;  Kyllonen, 
Lohman,  6t  Snow,  1984;  Pellegrino,  Mumaw,  &  Shute ,  1985). 

The  tasks  addressed  in  this  paper  concern  a  right-angled 
triangle  whose  vertical  height  was  150  units  on  a  computer 
presentation  screen,  and  whose  horizontal  side  adjacent  to  the 
right  angle  was  80,  100,  120,  or  140  units  (see  Figure  1).  The 
model  triangle  was  presented  on  the  top  half  of  the  screen. 
Immediately  below  it  was  a  second  triangle  whose  sides  were 
identical  in  size,  but  which  had  been  rotated  from  the  vertical  by 
40,  80,  120,  or  160  degrees.  This  target  was  either  an  exact 
match  to  the  original  or  a  mirror- image,  and  the  subject  was 
instructed  to  indicate  whether  the  two  triangles  were  the  same  or 
different.  Response  latency  and  correctness  were  recorded.  The 
stimulus  set  was  constructed  from  the  Cartesian  product  of  four 
factors;  side  lengths  (4),  rotations  (4),  identical  or  mirror- 
image  (2) ,  and  hypotenuse  of  the  model  triangle  left  or  right  of 
the  vertical  (2),  for  64  distinct  stimuli  in  total.  Because 
systematic  differences  are  found  routinely  between  patterns  on 
rotations  tasks  whose  correct  answer  is  "same"  compared  to  those 
for  which  the  correct  answer  is  "different"  (e.g.,  Cooper  6i 
Podgomy,  1976),  we  address  only  the  32  stimuli  in  which  the 
target  triangle  is  identical  to  the  model. 

[Figure  1  about  here] 

Subjects  were  male  recruits  to  the  British  Army,  between  18 
and  24  years  of  age,  undergoing  selection  and  allocation  in  the 
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Army  Personnel  Selection  Centre.  They  were  tested  throughout  by  a 
single  presenter  during  the  week  of  October  17,  1988.  They  were 
assigned  in  groups  of  approximately  40  subjects  by  daily  cohort  of 
recruits  to  one  of  three  experimental  groups.  All  groups  were 
presented  two  replications  of  the  entire  task  set,  but  they 
received  different  instructions.  Two  groups  started  under  the 
"standard  testing  condition"  described  below,  and  the  third  group 
was  additionally  shown  a  nonspatial  rule -based  strategy  for 
solving  the  tasks.  One  of  the  two  standard  groups  was  given  the 
rule-based  strategy  instruction  before  their  second  replication, 
but  because  only  data  from  the  first  replication  is  addressed 
here,  the  two  standard  groups  will  not  be  distinguished. 

The  standard  testing  condition  was  derived  from  the  customary 
way  of  demonstrating  this  class  of  rotational  tasks  to  naive 
subjects.  An  overhead  projector  was  used  to  display  two  cardboard 
right  triangles  vertically,  as  in  Figure  1,  creating  solid  black 
images  on  a  white  background.  The  bottom  triangle  was  rotated  to 
several  positions  to  show  that  it  could  be  "shuffled  around  on  the 
page"  to  match  the  top  one.  It  was  physically  lifted  to  show  that 
it  could  be  fitted  exactly  over  the  top  one.  By  similar  means, 
the  mirror  image  of  the  top  triangle  was  shown  to  be  different 
from  the  top  one,  no  matter  how  it  was  moved  about  in  the  plane. 
The  same  demonstration  was  given  for  both  the  hypotenuse -left  and 
hypotenuse -right  versions  of  the  original  stimulus.  This 
introduction  was  devised  to  encourage  the  subjects  to  use  a  mental 
rotation  strategy  for  problem  solution,  a  strategy  described  by 
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Shepard  and  Meltzer  (1971)  as  "isomorphic."  Under  this  strategy, 
degree  of  rotation  is  a  primary  determiner  of  task  difficulty,  in 
terms  of  both  response  latency  (Shepard  &  Meltzer,  1971)  and 
accuracy  (Tapley  &  Bryden,  1977). 

The  second  experimental  condition  was  devised  to  encourage 
the  use  of  an  "analytic"  or  "rule-based"  strategy.  Such 
strategies  employ  procedural  learning  rules  rather  than  mental 
visualization  to  determine  correct  answers.  Subjects  were  shown 
how  to  judge  whether  the  target  was  identical  or  mirror- image  of 
the  model  by  attending  to  a  specific  feature  of  the  triangle, 
namely  the  length  of  the  side  clockwise  adjacent  to  the  right 
angle.  The  triangles  are  different  if  one  encounters  the  long 
side  of  one  triangle  but  the  short  side  of  the  other.  It  can  be 
hypothesized  that  under  this  strategy,  difficulty  is  nearly 
independent  of  the  degree  of  rotation,  but  depends  primarily  on 
the  saliency  of  the  key  feature;  that  is,  the  strategy  should  be 
more  difficult  to  implement  as  the  right  triangle  becomes  more 
nearly  isosceles. 

The  rule  was  derived  by  asking  British  Naval  Engineering 
cadets  at  the  Royal  Naval  Engineering  College  at  Plymouth  how  they 
habitually  solved  such  tasks.  While  some  replied  they  knew  just 
by  looking  at  them  (i.e.,  their  strategies  were  not  consciously 
available),  others  said  they  never  rotated  the  objects  themselves; 
they  instead  moved  their  gaze  around  the  model  from  a  fixed  point 
on  it,  noting  the  presence  or  absence  of  some  salient  feature. 

They  then  found  the  same  starting  point  on  the  target  and  checked 
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for  the  presence  or  absence  of  the  feature.  If  it  was  there,  the 
target  had  to  be  the  same  as  the  model.  If  not,  it  had  to  be 
different . 

Data  from  244  subjects  were  initially  made  available  for  this 
study.  The  analyses  reported  here  concern  the  196  remaining  after 
trimming  those  with  the  highest  and  the  lowest  10-percent  of 
within  task- type  variance,  as  pooled  over  the  16  rotation/side - 
length  pairs,  so  as  to  leave  the  remaining  80-percent  nearly 
homoscedastic  with  respect  to  this  component  of  variation.  Of 
these,  131  had  not  been  instructed  in  the  rule  and  65  had.  Note 
that  the  2:1  ratio  reflects  equal  trimming  of  noninstructed  and 
instructed  subjects. 

Figures  2  and  3  plot  median  latencies  to  "same"  tasks  against 
coded  angular  displacement  and  side  length,  respectively. 
Throughout  the  paper  we  work  with  natural  logarithms  of  response 
times,  standardized  over  all  tasks  and  subjects.  Figure  2  shows 
that  average  response  latency  tends  to  increase  linearly  with 
angular  displacement,  which  is  coded  as  -1.5,  -.5,  .5,  and  1.5 
along  the  x-axis.  The  plot  symbols  are  coded  side  lengths,  from 
shortest  (most  acute)  to  longest  (most  nearly  isosceles)  as  -2.25, 
-1.25,  .75,  and  2.75.  Coding  side  length  in  these  unequally 
spaced  intervals  produces  the  nearly  linear  relationship  shown  in 
Figure  3,  where  the  x-axis  gives  coded  side  length  and  plot 
symbols  give  coded  angular  displacements.  Table  1  gives  three 
representative  subjects'  observed  means,  averaging  over  left-  and 
right-hypotenuse  tasks  within  rotation  and  side-length. 
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[Figures  2  and  3  about  here] 

[Table  1  about  here] 

The  Model 

A  mixture  model  for  the  data  described  above  is  laid  out  in 
two  phases.  First  is  the  response  model,  which  concerns  the 
distribution  of  response  latencies  conditional  on  choice  of 
strategy  and  proficiency  in  using  that  strategy.  Second  is  the 
population  model,  which  concerns  the  proportions  of  subjects  that 
employ  the  various  strategies  and  the  distributions  of  proficiency 
within  strategy  classes.  The  generic  model  described  below  is  for 
a  mixture  of  K  strategy  groups;  a  single  strategy  can  be  modelled 
by  setting  K  to  one.  After  presenting  the  model,  we  give  the 
results  of  fitting  three  single -strategy  models  and  a  mixture  of 
two  strategies. 

The  Response  Model 

Suppose  that  K  potential  strategies  are  available  to  solve 
tasks  in  a  test,  and  each  subject  uses  one  strategy  exclusively  on 
all  tasks.  Each  subject  will  be  characterized  by  two  vector¬ 
valued  unobservable  variables.  The  first  is  the  indicator 

variable  ^  -  (<£.. . <f>v)  ,  where  <f>,  takes  the  value  1  if  Strategy  k 

is  the  one  the  subject  uses,  0  if  not.  The  second  is 
8  -  where  0^  *-s  proficiency  under  Strategy  k.  Only 

one  of  the  elements  of  8  plays  a  role  in  producing  the  data 
observed  from  a  given  subject,  namely  the  one  for  which 
Task  j  is  characterized  by  the  vector  of  difficulty  parameters 
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{) .  -  (£... . P-v),  where  determines  the  difficulty  of  Task  j 

~J  J 1  J K  J  K 

for  subjects  who  employ  Strategy  k. 

We  work  with  a  log  normal  distribution  for  the  response 

latency  t^  of  a  subject  to  Task  j,  given  that  Strategy  k  is 

employed.  Defining  x.  as  fn  t.  and  6^  as  a  scale  parameter  of  the 

J  J 

log-normal  density  pertaining  to  Strategy  k,  the  response  model  is 
given  by 


f<*jl t-l'iyV  -  J  W'k-VV 


where 


:<xjl  WV  ‘  'jtl  “P  "  ^2  [Xj-(V^k>l2  ’  '  [2i 


Thus  [2]  gives  the  density  for  log  response  time  to  Task  j  for  a 
subject  employing  Strategy  k,  with  proficiency  0^  under  that 
strategy.  The  difference  *-s  the  ®aan  and  6^  is  the 

standard  deviation.  Note  that  lower  values  of  8  signify  faster 
expected  response  times  (i.e.,  more  proficiency  on  the  part  of  the 
subject),  while  higher  values  imply  slower  response  times.  Lower 
values  of  f)  signify  slower  expected  response  times  (greater 
difficulty  on  the  part  of  the  task),  while  higher  values  imply 
faster  response  times.  The  product  over  k  that  appears  in  [1] 
serves  merely  to  select  the  f^  that  applies,  since  ^  is  1  in  this 
case  and  0  in  all  others.  In  the  sequel  f^(x^  |  5^)  will  be 

abbreviated  at  times  as  f^(Xj|S^),  suppressing  the  dependence  on 
0jk  «nd  Sk. 
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Equations  [1]  and  [2]  posit  individual  differences  among 

subjects  as  to  strategy  selection  and  speed-within- strategy . 

Individual  differences  corresponding  to  slopes  might  also  be 

entertained,  but  to  do  so  here  increases  complexity  without  adding 

insight  into  the  inferential  approach.  We  note  in  passing, 

however,  that  recent  studies  suggest  that  at  least  some  of  the 

slope  differences  among  individuals  found  in  standard  analyses  may 

be  related  to  the  use  of  different  strategies  (Carter,  Pazak,  & 

Kail,  1983;  Just  &  Carpenter,  1985). 

Psychological  theory  about  what  makes  tasks  easy  or  hard 

under  various  strategies  appears  in  the  form  of  models  for  the 

ft.,  's.  Following  Scheiblechner  (1972)  and  Fischer  (1973),  let  q 
J  K  “J  K 

be  a  known  vector  of  coefficients  expressing  the  extent  to  which 
Task  j  exhibits  each  of  features  that  determine  task  difficulty 
under  Strategy  k;  let  be  a  parameter  vector,  also  of  length 
that  conveys  the  influence  of  those  features.  Assuming  a  linear 
relationship  between  task  features  and  task  difficulty, 


^jk  ”  qjkm  Qkm  "  9jk  ~k 


For  the  rotational  strategy,  for  example,  angular 
displacement  is  a  major  determiner  of  difficulty.  Tasks  rotated 
to  the  four  equally- spaced  displacements  will  have  q  values  for 
this  feature  coded  as  -1.5,  -.5,  .5,  or  1.5.  The  corresponding 
element  of  a  would  indicate  the  incremental  difficulty  associated 
with  an  increase  of  one  additional  unit  of  angular  displacement. 
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For  a  strategy  under  which  side  length  is  posited  to  contribute  to 
difficulty,  tasks  will  have  q  values  of  -2.25,  -1.25,  .75,  or  2.75 
that  indicate  the  saliency  of  this  key  feature.  The  corresponding 
element  of  a  would  reflect  the  additional  difficulty  resulting 
from  an  additional  increment  toward  being  isosceles. 

Task  responses  are  assumed  to  be  independent  given  a,  8  and 
<f>.  Letting  x-(x^ . x^)  be  a  vector  of  log  latencies  to  n  tasks, 

P(x|0k,*k-l,ak,5k)  -  n  fk(x  |*k)  -  fk(x|*k)  , 

j  J 

so  that 

*k 

p(x|fi,^,a,£)  -  n  fk(x|*k)  K  .  [4] 

k 


The  Population  Model 

Suppose  that  subjects  are  a  representative  sample  from  a 
population  in  which  the  proportion  employing  Strategy  k  is  xk, 
with  0<irk<l.  Denote  by  n  the  vector  ,  .  .  .  ,ir^)  .  Denote  by 
gfc(^k^  t*ie  density  function  of  #k  for  members  of  Class  k.  We 
assume  normal  distributions  (although  other  distributions  or  even 
nonparametric  approximations  could  be  used),  so  that  6k^k^k’ak^ 
has  the  form 


sk^kl^k’CTk) 


Jlir  a. 


exp 


(i 


(VV 


For  brevity,  denote  the  population  distribution  parameters  by  T  - 

(2i . 2K>  "  °Val . VV- 
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Estimation 


Equation  [4]  is  the  conditional  probability  of  a  response 
pattern  x,  or  the  probability  of  observing  x  from  a  subject  having 
particular  values  of  6  and  <£.  Assuming  the  population  model 
described  above,  the  probability  of  observing  x  from  an  examinee 
selected  at  random,  or  the  marginal  probability  of  x,  is  given  as 

s  \  S  fk(-,*k’2k,5k)  6k(^k|2k)  dV  151 

k 

Let  x  -  (x^,...,x^)  be  the  response  matrix  of  a  random  sample  of  N 
subjects  to  n  tasks.  A  realization  of  x  induces  the  marginal 
likelihood  function  for  a,  6,  w,  and  T  as  the  product  over 
subjects  of  factors  like  [5]: 

N 

L(q ,  6  , it , r | x)  “  II  p(x.  |cr,  6  , >r,r)  .  [6] 

i-1 

We  refer  to  a,  5,  w,  and  T  as  the  structural  parameters  of  the 
problem  Their  number  remains  constant  as  N  increases.  The 
incidental  parameters  6  and  <t> ,  whose  numbers  increase  in  direct 
proportion  to  N,  have  been  eliminated  by  marginalizing  over  their 
respective  distributions.  Marginal  maximum  likelihood  (MML) 
estimation  finds  the  values  of  the  structural  parameters  that 

AAA  A 

maximize  [6],  say  a,  S,  n,  and  T.  The  Appendix  gives  a  numerical 
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solution  for  the  present  problem  using  Dempster,  Laird,  and 
Rubin's  (1977)  EM  algorithm.  The  solution  is  for  data  in  which 
strategy  use  is  not  known  with  certainty  for  any  subject,  as  was 
the  case  in  the  present  study.  If  strategy-use  information  is 
available  for  some  subjects,  however,  as  from  follow-up  interviews 
with  a  subsample,  estimates  of  structural  parameters  can  be 
improved- -sometimes  dramatically- -by  exploiting  it  (Titterington, 
Smith,  &  Makov,  1985,  Section  4.2). 

Once  MML  estimates  of  structural  parameters  have  been 
obtained,  one  can  obtain  empirical  Bayesian  approximations  of 
probabilities  of  class  membership  for  any  examinee,  and  estimates 
of  0 k  conditional  on  membership  in  any  class  k.  If  the  structural 
parameters  were  known  with  certainty,  the  posterior  density  for 
Subject  i  would  be 

p(ek’^k"1l~i,2,£,H’EJ 

*k  fk^~i^k’2k’6k^  gk^*k^k^ 

Vil'VW  6h<»hISh>  d#h 

n 

An  empirical  Bayes  approximation  substitutes  MLEs  for  the 
structural  parameters.  The  posterior  probability  that  Subject  i 
employed  Strategy  k,  denoted  P  is  approximated  as 

A  A  A  A 

pik  "  p(V1|-i)  “  f  p(W1l-i,2’£’2’-)  d*k  •  [8] 

Conditional  on  membership  in  Class  k,  the  posterior  expectation 
and  variance  of  0^  are  approximated  as 


var(*k|,ik-l,Xi)  - 


_  A  A  A  A 

J  ^  P(tfk^k“1|5i-2'£-J'£)  d*k 


[10] 


Single -Strategy  Solutions 

This  section  uses  single -strategy  solutions  to  illustrate  Q 
matrices,  obtain  baseline  likelihood  statistics,  calculate  task- 
difficulty  estimates  under  different  models,  and  give  a  feel  for 
the  data  by  looking  more  closely  at  the  subjects  introduced  in 
Table  1.  These  solutions  assume  not  only  that  a  subject  applies 
the  same  strategy  on  all  tasks,  but  that  all  subjects  are  applying 
the  same  strategy.  Three  such  solutions  are  considered: 

1.  Rule-based  (difficulty  depends  on  side  length  only), 

2.  Mental  rotation  without  side -length  effects  (difficulty 
depends  on  degree  of  rotation  only) ,  and 

3.  Mental  rotation  with  side-length  effects  (difficulty  depends 
on  both  side  length  and  degree  of  rotation) . 

Side -Length  Only 

Suppose  that  all  subjects  followed  a  rule-based  strategy, 
under  which  difficulty  is  determined  by  side  length  alone.  Under 
this  scenario,  the  only  systematic  sources  of  variation  in 


response  times  are  subjects'  overall  proficiencies  and  tasks' 
differing  side  lengths.  Proficiency  is  denoted  by  the  univariate 
variable  8 ,  assumed  to  follow  a  normal  distribution  whose  mean  /i 
and  standard  deviation  a  are  to  be  estimated.  Task  difficulty  is 
given  as  0 ^  a^,  where  is  the  coded  side -length  of  task  j 

(-2.25,  -1.25,  .75,  or  2.75),  and  is  the  change  in  difficulty 
associated  with  an  additional  increment  in  side  length  toward 
isosceles.  Finally,  S  is  the  standard  deviation  of  log  response 
time  within  task  type  and  subject. 

The  first  panel  of  Table  2  gives  the  MLEs  of  the  structural 
parameters.  The  value  of  -2  log  likelihood,  or  -2  in  2  p(x^) , 
also  appears,  which  will  be  used  to  compare  the  fit  of  alternative 
models.  The  estimate  of  is  -.124,  which  translates  into 
expected  increases  in  response  latency  as  side  length  increases. 
The  first  panel  of  Table  3  gives  the  resulting  0s,  with  higher 
values  corresponding  to  faster  response  times  and  lower  numbers 
corresponding  to  slower  ones.  The  modelled  standardized  log 
latency  of  a  particular  subject  on  a  particular  task  type  under 
this  model  would  be  obtained  by  subtracting  the  task's  0  value 
from  that  subject's  8  value- -  thereby  maintaining  for  all  subjects 
the  pattern  of  increasing  difficulty  with  increasing  side  length 
without  effects  of  degree  of  rotation. 

[Insert  Tables  2  and  3  about  here] 

This  pattern  is  illustrated  with  the  subject  whose  observed 
means  data  appear  in  the  second  panel  of  Table  1.  The  Bayes 
estimate  8  is  -.317,  a  measure  of  overall  proficiency.  Combining 
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6  with  the  0s  gives  the  modelled  latencies  in  the  second  panel  of 
Table  4.  The  third  panel  gives  residuals,  which  indicate  faster 
averages  than  expected  in  some  cells,  slower  in  others,  but  no 
systematic  trend  in  average  residuals  in  either  the  side-length  or 
rotation  margins.  (The  residuals  do  not  sum  to  zero  because  the 
expectations  were  based  on  a  Bayesian  proficiency  estimate,  as 
opposed  to,  say,  least-squares  within  subject.)  Table  5  gives 
residuals  for  all  three  sample  subjects.  Note  the  strong  trend 
related  to  rotation  for  the  last  subject. 

[Insert  Tables  4  and  5  about  here] 

Rotation  Only 

Even  though  the  side-length  model  fits  the  data  of  subjects 
like  the  second  fairly  well,  the  familiar  relationship  between 
difficulty  and  degree  of  rotation  appears  in  the  residuals  of 
subjects  like  the  fifth.  An  alternative  simple  model  has 
difficulty  depend  on  rotation  only,  neglecting  the  possible  impact 
of  differences  in  side -length.  Under  this  model,  0  -  a 
where  is  the  coded  rotation  of  Task  j  (-1.5,  -.5,  .5,  or  1.5) 
and  is  the  impact  on  difficulty  of  an  additional  increment  in 
rotation.  The  results  of  fitting  this  model  appear  in  the  second 
panel  of  Table  2.  The  MLE  of  is  -.108,  signalling  increasing 
difficulty  as  the  degree  of  rotation  increases .  The  resulting  0s 
are  in  the  second  panel  of  Table  3,  showing  patterns  of  expected 
difficulty  that  depend  on  rotation  alone. 

The  value  of  -2  log  likelihood  is  14907.78.  To  compare  the 
fit  of  alternative,  possibly  nonnested,  models  to  the  same  data 
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set,  Akaike  (1985)  calculates  the  index  AIC  ("an  information 
criterion")  for  each  model:  -2  log  likelihood  plus  twice  the 
number  of  parameters  estimated.  The  model  with  the  smaller  AIC  is 
preferred.  AIC  for  rotation-only  is  14907.78  +  2x4  -  14915.78. 

AIC  for  side -length  only,  which  also  has  four  parameters,  is 
14432 .40- -smaller  by  483.  The  side-length  only  model  fits  better 
than  the  rotation  only  model. 

Subject  6  estimates  in  this  model  combine  with  the  /3s  to  give 
expected  response  latencies  that  maintain  the  same  relative 
relationships  seen  in  the  f) s,  but  vary  as  to  overall  speed. 
Subtracting  such  expectations  calculated  with  8s  from  the  observed 
data  gives  the  residuals  shown  in  Table  6  for  the  sample  subjects. 
The  third  subject's  strong  linear  trend  in  residuals  associated 
with  rotation  has  beer,  dampened  considerably,  but  a  reverse  trend 
in  the  rotation  margin  has  been  introduced  for  the  first  two 
subjects.  Note  also  the  tendency  toward  a  consistent  ordering  in 
the  side-length  residuals. 

[Table  6  about  here] 


Rotation  and  Side -Length 

A  final  single-strategy  model  incorporates  main  effects  for 
both  rotation  and  side -length:  /S^  ”  QX  +  ^2  Q2’  w*iere  t*ie  9s 
and  as  have  the  same  meanings  as  in  the  preceding  models.  This 
incorporates  a  more  elaborate  scenario  for  mental  rotation, 
allowing  for  latency  to  increase  not  only  with  degree  of  rotation, 
but  with  the  difficulty  of  comparison  after  rotation  is  complete 
(Cooper  &  Podgorny,  1976).  Parameter  estimates  and  -2  log 
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likelihood  appear  in  the  third  panel  of  Table  2.  Because  of  the 

A 

orthogonal  experimental  design,  as  are  identical  to  those  of  the 
previous  single -feature  models.  AIC  for  this  model  is  14258.72, 
smaller  than  the  side-length  AIC  by  173  and  smaller  than  the 
rotation  AIC  by  661.  The  model  incorporating  both  task  features 
thus  fits  better  than  both  models  with  a  single  feature  only,  even 
accounting  for  the  additional  parameters  being  estimated.  The 
resulting  f)s  appear  in  the  third  panel  of  Table  3. 

The  residuals  for  the  sample  subjects  are  in  Table  7. 

Because  of  the  orthogonal  design,  the  rotation  margins  of  the 
residual  tables  under  this  model  match  the  corresponding  margins 
under  the  rotation  only  model,  and  the  side-length  margins  match 
those  from  the  side-length  only  model. 

[Table  7  about  here] 

A  Two- Strategy  Solution 

This  model  posits  a  mixture  of  two  types  of  subjects:  those 
employing  the  rule-based  strategy,  whose  response  patterns  can  be 
largely  captured  with  the  side -length  only  model,  and  those 
employing  mental  rotation,  whose  responses  depend  on  both  side- 
length  and  degree  of  rotation.  There  are  ten  parameters  to 
estimate  in  this  model:  the  relative  proportion  of  strategy  use  in 
the  sample;  a  mean,  standard  deviation,  and  within  task- type 
standard  deviation  for  each  strategy;  an  a  for  side -length  for  the 
rule-based  strategy  component;  and  two  as,  one  for  side-length  and 
one  for  rotation,  for  the  rotation  strategy  component.  The 
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estimates  appear  in  the  final  panel  of  Table  2.  AIC  improves  over 
the  rotation  and  side-length  single-strategy  model  by  72. 

Two  distinct  sets  of  f)s  result  in  this  model,  one  for  each 
strategy.  These  appear  in  the  last  two  panels  of  Table  3.  The 
side-only  component,  like  the  side-length  only  single  strategy 
panel,  shows  only  a  side-length  effect.  Interestingly,  it  is 
smaller  than  the  corresponding  effect  in  the  side- length  single¬ 
strategy  model.  The  "side-length  and  rotation"  component  shows 
both  effects.  Compared  to  the  corresponding  single  strategy 
solutions,  both  the  side-length  and  the  rotational  effects  are 
stronger.  The  interpretation  would  be  that  decreasing  the 
distinctness  of  the  salient  feature  of  the  stimulus  hampers 
subjects  employing  rotation  more  than  those  employing  the  rule. 

The  point  of  mixture  modelling  is  that  we  do  not  know  with 
certainty  which  subjects  are  employing  which  strategy.  One  of  the 
structural  parameters  is  the  proportion  using  each;  MLEs  are  57- 
percent  for  the  rule -based  strategy  and  43 -percent  for  the 
rotational  strategy.  These  values  can  be  expressed  as  the 
averages  of  subjects'  posterior  probabilities  of  being  in  one 
strategy  group  or  the  other  (Equation  8).  The  histograms  in 
Figure  4  show  subjects'  posterior  probabilities  for  the  rule- 
based  strategy.  Most  have  probabilities  below  .2  or  above  .8, 
indicating  fairly  good  separation  of  the  components  of  the 
mixture.  To  put  this  in  the  context  of  the  statistical  literature 
on  mixtures  (e.g.,  Titterington  et  al.,  1985),  Figure  5  gives  the 
histogram  of  a  closely  matching  "mixture  of  homoscedastic  Gaussian 
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components"  problem.  The  mixing  probability  is  .55  and  the 
distance  between  the  means  of  the  components  is  2.2  standard 
deviations .  The  information  about  the  mixing  proportion  is  about 
half  what  it  would  be  if  component  membership  were  observable 
(Hill,  1963).  Figure  6  breaks  the  information  from  Figure  4  down 
by  whether  subjects  were  shown  the  rule.  The  proportion  of 
instructed  subjects  whose  responses  are  strongly  allied  with  the 
rule-based  strategy  is  substantially  higher- -nearly  half,  as 
compared  to  just  a  fourth  of  those  not  instructed. 

[Insert  Figures  4-6  about  here] 

Table  8  gives  posterior  probabilities  and  residuals  for  the 
sample  subjects.  As  suggested  by  patterns  of  residuals  from  the 
single  strategy  models,  the  posterior  probabilities  of  Subjects  1 
and  2  are  concentrated  on  Strategy  1,  the  rule-based  strategy  (Ps 
of  .98  and  .70),  and  that  of  Subject  3  is  concentrated  on  the 
rotational  strategy  (P  nearly  1.00).  The  main  factor  that 
determines  strategy  assignment  is  the  presence  of  a  linear  trend 
related  to  rotation.  Of  secondary  importance  is  the  strength  of 
the  trend  related  to  side  length,  with  stronger  trend  being 
associated  with  the  rotational  strategy.  The  high  probability  of 
the  rule-based  strategy  for  Subject  1  is  a  consequence  of  both  a 
lack  of  trend  for  rotation  and  a  weak  trend  for  side  length  in  the 
observed  data  (see  Table  1). 

For  a  decision-making  problem  that  depends  on  identifying  a 
subject's  strategy  use,  the  strategy  with  the  higher  posterior 
probability  is  the  choice.  For  a  problem  that  depends  on 
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predicting  future  performance,  the  optimal  prediction  is  a 
mixture.  For  Subject  2,  for  example,  predictions  would  be 
calculated  under  both  strategies,  using  the  task  parameters  and 
his  posterior  6  distribution  within  each;  these  distinct 
predictions  would  be  averaged  with  weights  of  .70  for  the  rule- 
based  strategy  and  .30  for  the  rotational  one.  The  predictions 
for  the  residuals  in  Table  8  were  calculated  in  this  manner,  using 
conditional  within- strategy  posterior  means  for  8s. 

[Insert  Table  8  about  here] 

Discussion 

This  presentation  was  meant  to  demonstrate  how  a  mixture - 
model  approach  can  be  incorporated  into  test  theory  to  deal  with 
different  problem  solving  approaches.  We  readily  concede  that  the 
psychological  model  for  the  visualization  tasks  is  overly 
simplistic.  There  are  some  ways  it  could  be  made  more  realistic 
while  preserving  the  assumption  that  a  subject  maintains  the  same 
strategy  throughout  observation.  These  include  modelling 
subprocesses,  relaxing  distributional  assumptions,  or 
incorporating  individual  difference  terms  for  residual  variances 
or  sensitivity  to  rotational  angles.  But  more  substantial  steps 
toward  reality  lie  beyond  this  framework. 

It  must  first  be  admitted  that  the  two  strategies  discussed 
here  do  not  exhaust  the  variety  of  approaches  that  subjects  bring 
to  bear  upon  such  tasks.  They  may  be  viewed  as  archetypes,  one  or 
the  other  of  which  may  be  sufficiently  close  to  a  given  subject's 
data  to  provide  a  serviceable  guide  to  selection  or  instruction. 
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In  applied  work,  analyses  of  subjects'  fit  and  patterns  of 
residuals  will  be  important  to  flag  patterns  that  are  not 
modelled,  either  for  special  consideration  if  they  are  few  in 
number,  or  for  incorporation  into  the  model  if  they  are  recurrent. 

Perhaps  more  importantly,  we  must  consider  the  possibility  of 
strategy  switching,  or  the  violation  of  the  assumption  that  a 
subject  follows  the  same  strategy  on  all  tasks.  This  assumption 
may  be  reasonable  if  strategy  is  defined  through  knowledge 
structures,  so  that  response  patterns  associated  with  advanced 
structures  are  not  accessible  to  learners  in  earlier  stages.  An 
example  of  this  type  is  Siegler's  analysis  (1981)  balance  beam 
tasks,  where  competence  is  increased  by  adding  rules  to  a 
repertoire  in  a  largely  predictable  order.  It  may  also  be 
reasonable  if  strategies  associated  with  alternative  mental  models 
of  a  domain  are  unlikely  to  coexist  in  a  given  subject. 

The  same  assumption  is  less  reasonable  in  a  domain  where 
alternative  strategies  are  available  to  the  same  subject,  so  that 
the  conditions  and  the  frequency  of  strategy  switching  become 
additional  sources  of  individual  difference.  Kyllonen,  Lohman, 
and  Snow  (1984)  note  increasing  use  of  non-spatial  strategies  as 
mental  rotation  tasks  become  more  difficult.  A  more  appropriate 
model  in  this  case  would  characterize  mixtures  within  subjects; 
not  just  that  a  subject  followed  Strategy  A  or  Strategy  B,  but 
that  she  used  Strategy  A  on  30-percent  of  the  tasks  and  Strategy  B 
on  70-percent- -and  that  the  characteristics  of  tasks  that  were 
relevant  to  her  strategy  choice  were  such-and-such.  Given  the 
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vicissitudes  of  even  the  straight  mixture  model,  it  is  clear  that 
modelling  mixtures  within  subjects  will  require  richer  information 
than  the  familiar  correctness  or  response  times ,  perhaps  in  the 
form  of  response  protocols,  intermediate  products,  or  physical 
measures  such  as  eye  movement  (as  in  Just  &  Carpenter,  1976). 

It  would  appear  that  mixture  models  of  the  type  illustrated 
here  have  most  promise  in  contexts  where  the  qualitative 
distinctions  among  persons  are  relatively  few  in  number,  stable 
during  the  period  of  observation,  and  distinguishable  in  terms  of 
their  implications  for  observable  behavior.  We  have  demonstrated 
by  example  that  the  calculations  in  this  case  are  tractable  and 
the  mixtures  can  be  resolved  satisfactorily.  Future  work  will 
focus  on  a  more  general  computational  scheme  and  a  broader  variety 
of  more  educationally  relevant  applications. 
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Appendix:  Estimating  Structural  Parameters  via  the  EM  Algorithm 

Equation  [5]  is  an  "incomplete  data"  density  function  of  the 
form  addressed  by  Dempster,  Laird,  and  Rubin  (1977)  in  "Maximum 
likelihood  from  incomplete  data  via  the  EM  algorithm."  Estimating 
the  structural  parameters  would  be  straightforward  if  values  of 
the  latent  variables  6  and  <f>  could  be  observed  from  each  subject 
along  with  his  or  her  response  vector  x;  this  would  be  a  "complete 
data"  problem.  The  EM  algorithm  maximizes  the  incomplete -data 
likelihood  [6]  iteratively.  The  E-step,  or  expectation  step,  of 
each  cycle,  calculates  the  expectations  of  the  summary  statistics 
that  the  complete-data  problem  would  require,  conditional  on  the 
observed  data  and  provisional  estimates  of  the  structural 
parameters.  The  M-step,  or  maximization  step,  solves  what  looks 
like  a  complete-data  maximum  likelihood  problem  using  the 
conditional  expectations  of  summary  statistics.  The  resulting 
maxima  for  the  structural  parameters  are  improved  estimates  of  the 
incomplete-data  solution,  and  serve  as  input  to  the  next  E-step. 

We  employ  the  variation  of  the  EM  algorithm  suggested  by 
Mislevy  and  Verhelst  (in  press)  to  estimate  the  parameters  of 
mixtures  of  psychometric  models.  The  integration  that  appears  in 
[5]  is  approximated  by  summation  over  a  fixed  grid  of  points.  The 
E-step  calculates,  for  each  examinee,  conditional  probabilities  of 
belonging  to  each  component  of  the  mixture  (i.e.,  strategy  class) 
and,  conditional  on  component  membership,  the  probabilities  that  6 
takes  various  grid-point  values.  The  grid  points  play  the  role  of 
weighted  pseudo-data  points  in  the  M-step.  Numerical  integration 
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could  be  avoided  in  the  special  case  considered  above,  but  the 
approach  described  below  also  applies  with  alternative  response 
models  and  distributional  forms. 


Solving  the  "Complete  Data"  Problem 
This  section  gives  the  ML  solution  for  a,  6,  x ,  and  T  that 
would  obtain  if  values  of  B  and  <f>  were  observed  for  each  subject 
along  with  x.  Among  the  sampled  subjects  from  Strategy  Class 
k,  some  number  distinct  values  of  6  would  be  observed,  say 

for  i-l,...,L,  .  Define  the  following  statistics: 

o  an  indicator  variable  that  takes  the  value  1  if  Subject 

i  is  in  Strategy  Class  k  and  has  proficiency  0^. 


o  N.  ,  the  number  of  examinees  observed  to  be  in  Class  k: 
k 


Nk  -  *  *ik  “  S  *  *iki  • 

i  l  i 


o  t*ie  num^er  °f  examinees  in  Class  k  with  0-0^ : 


Nki  -  l  Xiki  • 


o  xjk’  t*le  mean  $ -centered  log  latencies  for  Task  j  from 
subjects  in  Class  k: 


xjk  ”  Nk  ^  J  (xij'6ki;  Xiki 


s^,  ,  the  average  square  of  6 -centered  log  latincies  for  Task 
J  k  “ 

j  from  subjects  in  Class  k: 
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sjk  “  Nkx  *  J  (xij'eki)  Iiki  ■ 


The  complete  data  likelihood  for  a,  S,  ir ,  and  T  that  would  be 
induced  by  the  observation  of  x,  0,  and  £  can  be  written  as 


L  (a,{  ,Jr,r|x,  6  ,<f>)  - 


n  P(Nki!r)  n  p(Nki|Nk,a)  n  VSi^kr^k-V 

k  i  i 


whence  the  complete  data  log  likelihood 


i  <«.£.w.r|x,£.*>  -nk  log  irfc  E  Nki  log  Sk^kil^k’^k^ 


+  ?  I  Xiki  f  log  fk(xij  |0ki’-k’5k) 

1  k  i  j  J 


ML  estimation  for  the  complete  data  problem  proceeds  by  solving 
the  likelihood  equations ,  which  are  obtained  by  setting  to  zero 
the  first  derivatives  of  [A5]  with  respect  to  each  element  of 
(a, 6  ,ir,r)  .  That  is,  for  a  generic  element  u  of  (o, £,£,£), 


ai  (a, 6fw,r|x, $,<*>) 


For  elements  of  w,  one  must  impose  the  constraint  that  E»rk-1 , 
say  with  a  Lagrangian  multiplier.  One  obtains  a  closed  form 
solution  for  the  proportion  of  subjects  in  each  strategy  class: 
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*k  -  Nk/N  .  [A7 ] 


For  elements  of  the  population  parameter  vector  T,  the  ML 
estimates  in  the  normal  case  are,  for  k-1 . K, 


A 


[A8] 


and 


*2 

ak 


"k1  * 


[A9] 


For  the  parameters  of  the  task  response  model, 


A 


-  •  (W1  Sk5k  ■ 


where  -  [glk 


3nkJ 


and 


Sk  -  <i 


Ik' 


•xnk>' 


and 


[A10] 


6 


2 

k 


+  2  Vj* 


[All] 


Solving  the  "Incomplete  Data"  Problem 
The  likelihood  equations  in  the  incomplete -data  problem,  in 
which  6  and  are  not  observed,  are  the  first  derivatives  of  the 
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log  of  the  marginal  likelihood  function,  [6],  set  to  zero.  Under 
mild  regularity  conditions,  these  take  the  form  of  a  data-weighted 
average  of  the  complete-data  likelihood  equations  shown  as  [A6] . 
Letting  i(a, 5  ,ir,r|x)  -  in  L(a ,  5  ,  ir ,  T  j  x)  ,  the  incomplete-data 
likelihood  equation  for  a  generic  element  u  of  (ct,5,»r,r)  is 

8  i(o,fi,ir,r|X) 

0  _  - 

3u 

d£* (a,  6  ,jr,r|x,  0  — 1) 

0-s/ -  —  p(t,4 ,-i|x)  de  ,  [A12] 

k  3u 

where 

p(x  1 6  ,4>  ,a,5  .tt.D 

P(£,^|x)  -  n  [A13] 

i 

It  is  seen  in  [5]  that  evaluating  the  denominator  of  [A13] 
involves  integration  over  the  N(^,t7^)  distributions.  We 
approximated  these  integrals  via  Gaussian  quadrature.  Tabled 
values  are  obtained  for  L  points  ©^,...,0  ,  with  associated 
weights  W^,...,W^.  (We  used  L-80,  ranging  from  -3  to  +3). 
Calculation  proceeds  as  though  these  were  the  only  possible  values 
for  8;  indicates  whether  Subject  i  used  Strategy  k  and  had 

proficiency  8^. 

In  the  incomplete -data  problem,  neither  the  values  of  6 ^  nor 

6,  are  known,  so  neither  are  the  I.,  .  values  used  to  c.  lculate 
-i  iki 
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summary  statistics.  If  the  values  of  the  structural  parameters  F, 
it,  6,  and  a  were  known,  however,  it  would  be  possible  to  calculate 
the  expected  values  of  the  I^^s  given  x^s  as  follows: 


hki  ‘ 


*k  Ul  fk(Sil9l,Sk'tk>  sk<9(l~|,k) 
J’hSMr  fh<il'VW  ®h(W 

h  r 


[A14] 


In  the  E-step  of  the  EM  algorithm,  one  evaluates  [A14]  for 

each  i,  k,  and  i  using  provisional  estimates  of  a,  6,  it,  and  T. 

One  then  obtains  expectations  of  the  summary  statistics  defined  in 

~2 

[Al]-[A4],  say  N^,  N^,  x^,  and  s^.  The  grid  values  0^  in  the 

incomplete -data  solution  thus  correspond  to  the  observed  8  values 

in  the  complete  data  solution. 

In  the  M-step,  one  solves  facsimiles  of  the  complete  data 

'2 

likelihood  equations,  [ A7 ]  -  [ All ]  ,  with  N^,  N^,  Xj^>  and  in 
place  of  their  observed  counterparts.  Cycles  of  E-  and  M-steps 
continue  until  changes  are  suitably  small.  Convergence  to  a  local 
maximum  is  assured,  except  from  initial  values  that  lie  on 
boundaries  of  the  parameter  space  (e.g.,  tt^-0) .  Repeated 
solutions  from  different  starting  values  help  identify  the  global 
maximum.  Accelerating  methods  may  be  used  if  convergence  is  too 
slow. 


Equation  [A14]  will  be  recognized  as  an  application  of  Bayes 
theorem,  giving  the  posterior  probability  that  0^“®^  and 
after  observing  x^.  The  normalizing  constant  in  the  denominator 
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is  an  approximation  of  p(x^)  as  given  in  [5],  During  the  E-step, 
one  may  therefore  accumulate  the  sum  -2  £  log  p(x^)  to  track  the 
performance  of  improvement  in  fit  over  cycles,  to  examine  the  fit 
obtained  with  various  values  of  structural  parameters,  or  to 
compare  the  fit  of  alternative  models. 


Empirical  Bayes  Estimates  of  Examinee  Parameters 
The  numerical  approximations  employed  above  to  estimate 
structural  parameters  can  be  used  for  subsequent  empirical 
Bayesian  estimates  for  individual  subjects.  The  expectations  of 
the  indicator  variables  1.^  are  evaluated  via  [A14]  with  MML 
estimates  of  a,  S,  t ,  and  T.  The  empirical  Bayes  approximation  of 
probability  of  membership  in  Strategy  Class  k  is  given  as 


Pik  "  ^ik-1'^  -  \  Xiki  * 


[A15] 


Conditional  on  membership  in  Class  k,  the  posterior  expectation  of 
is  approximated  as 


-  E(0..  l^.-l.x.)  *  P.,  £9.  I..  . 

xk  ik'  ik  -i  ik  ^  i  iki 


[A16] 


and  the  corresponding  posterior  variance  is 


Var<<?ik^ik-1^i) 


£  e2  i 

i 


iki 


[A17] 
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Table  1 

Examples  of  Observed  Standardized  Mean  Log  Response  Times 


Subject  1 

Rotation 


Side  Length 

40 

80 

120 

160 

80 

-0 

.069 

0, 

.059 

-0.508 

-0. 

.676 

100 

-0 

.724 

-0. 

.483 

-0.242 

-0. 

.383 

120 

-0 

.591 

-0. 

.828 

0.406 

-0. 

993 

140 

-0. 

.288 

-0. 

,287 

-0.485 

-0. 

483 

Average 

-0 

.418 

-0. 

.385 

-0.207 

-0. 

634 

Within  Cell  Pooled  Standard  Deviation 


Subject  2 

Rotation 


Side  Length 

40 

80 

120 

160 

80 

-1.692 

-0.277 

-1.012 

-0.863 

100 

0.705 

0.514 

-0.450 

-0.871 

120 

-1.082 

-0.912 

0.224 

0.126 

140 

0.661 

0.015 

-0.316 

-0.035 

Average 

-0.352 

-0.165 

-0.389 

-0.411 

Within  Cell  Pooled  Standard  Deviation 


Subject  3 

Rotation 


Side  Length 

40 

80 

120 

160 

80 

-0.413 

-0.545 

-0.192 

-0.304 

100 

-1.699 

-1.849 

-0.108 

0.223 

120 

-1.186 

-0 .U19 

-0.189 

0.966 

140 

-0.451 

-0.815 

-0.525 

0.404 

Average 

-0.937 

-0.807 

-0.254 

0.322 

Average 

-0.299 

-0.458 

-0.502 

-0.386 

-0.411 

0.779 


Average 

-0.961 

-0.025 

-0.411 

0.081 

-0.329 

0.713 


Average 

-0.363 

-0.858 

-0.107 

-0.347 

-0.419 

0.687 


Within  Cell  Pooled  Standard  Deviation 


Table  2 

Parameter  Estimates  and  -2  Log  Likelihood 


Two  Stratee-' 

/  Model 

-2  log  likelihood  - 

14167.03 

Strategy 

ir 

4 

o 

6 

0(1) 

o(2) 

Rule 

.566 

0.099 

0.745 

0.668 

-0.104 

Rotation 

.434 

-0.129 

0.455 

0.753 

-0.151 

-0.219 

Table  3 

Task  Difficulty  Parameters 


Side  Length  Only  Model 

Rotation 


Side  Length 

40 

80 

120 

160 

80 

0.279 

0.279 

0.279 

0.279 

100 

0.155 

0.155 

0.155 

0.155 

120 

-0.093 

-0.093 

-0.093 

-0.093 

140 

-0.341 

-0.341 

-0.341 

-0.341 

Rotation  Only  Model 

Rotation 


Side  Length 

40 

80 

120 

160 

80 

0.162 

0.054 

-0.054 

-0.162 

100 

0.162 

0.054 

-0.054 

-0.162 

120 

0.162 

0.054 

-0.054 

-0.162 

140 

0.162 

0.054 

-0.054 

-0.162 

Side  Length  and  Rotation  Model 


Rotation 

Side  Length 

40 

80 

120 

160 

80 

0.441 

0.333 

0.225 

0.117 

100 

0.317 

0.209 

0.101 

-0.007 

120 

0.069 

-0.039 

-0.147 

-0.255 

140 

-0.180 

-0.287 

-0.395 

-0.503 

(continued) 


Table  3 ,  continued 
Task  Difficulty  Parameters 


Two  Strategy  Model:  Side  Length  Only  Component 

Rotation 


Side  Length 

40 

80 

120 

160 

80 

0.234 

0.234 

0.234 

0.234 

100 

0.130 

0.130 

0.130 

0.130 

120 

-0.078 

-0.078 

-0.078 

-0.078 

140 

-0.286 

-0.286 

-0.286 

-C  ' 

Two  Strategy 

Model : 

Side  Length  and  Rotation  Component 

Rotation 

Side  Length 

40 

80 

120 

160 

80 

0.667 

0.448 

0.229 

0.010 

100 

0.517 

0.298 

0.079 

-0.140 

120 

0.215 

-0.003 

-0.222 

-0.441 

140 

-0.086 

-0.305 

-0.524 

-0.742 

140  -0.086 


Table  4 

Modelling  Subject  2  under  Side  Length  Only 


Observed  Log  Response  Times 

Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

-1.692 

-0.277 

-1.012 

-0.863 

-0.961 

100 

0.705 

0.514 

-0.450 

-0.871 

-0.025 

120 

-1.082 

-0.912 

0.224 

0.126 

-0.411 

140 

0.661 

0.015 

-0.316 

-0.035 

0.081 

Average 

-0.352 

-0.165 

-0.389 

-0.411 

-0.329 

e  -  -.317 

Modelled  Loe  Response 

Times 

Rotation 

Side  Length 

40 

80 

120 

160 

Average 

80 

-0.599 

-0.599 

-0.599 

-0.599 

-0.599 

100 

-0.475 

-0.475 

-0.475 

-0.475 

-0.475 

120 

-0.227 

-0.227 

-0.227 

-0.227 

-0.227 

140 

0.021 

0.021 

0.021 

0.021 

0.021 

Average 

-0.320 

-0.320 

-0.320 

-0.320 

-0.320 

Residuals 

Rotation 

Side  Length 

40 

80 

120 

160 

Average 

80 

-1.096 

0.319 

-0.416 

-0.267 

-0.365 

100 

1.177 

0.986 

0.021 

-0.399 

0.446 

120 

-0.859 

-0.689 

0.447 

0.350 

-0.188 

140 

0.636 

-0.010 

-0.341 

-0.060 

0.056 

Average 

-0.035 

0.152 

-0.072 

-0.094 

-0.013 

Root  Mean  Square  Error  0.619 


Table  5 

Residuals  from  Side  Length  Only  Model 


Subject  1  (8  -  -.40) 

Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

0.605 

0.733 

0.167 

-0.002 

0.376 

100 

-0.174 

0.067 

0.309 

0.168 

0.092 

120 

-0.289 

-0.526 

0.708 

-0.691 

-0.199 

140 

-0.234 

-0.233 

-0.431 

-0.429 

-0.332 

Average 

-0.023 

0.011 

0.188 

-0.238 

-0.016 

Root  Mean  Square  Error  0.426 


Subject  2  ( 8  -  -.32) 

Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

-1.096 

0.319 

-0.416 

-0.267 

-0.365 

100 

1.177 

0.986 

0.021 

-0.399 

0.446 

120 

-0.859 

-0.689 

0.447 

0.350 

-0.188 

140 

0.636 

-0.010 

-0.341 

-0.060 

0.056 

Average 

-0.035 

0.152 

-0.072 

-0.094 

-0.013 

Root  Mean  Square  Error  0.619 


Subject  3  ( 8  -  -.40) 

Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

0.269 

0.138 

0.490 

0.379 

0.319 

100 

-1.140 

-1.291 

0.450 

0.781 

-0.300 

120 

-0.877 

0.290 

0.121 

1.276 

0.203 

140 

-0.389 

-0.753 

-0.463 

0.465 

-0.285 

Average 

-0.534 

-0.404 

0.149 

0.725 

-0.016 

Root  Mean  Square  Error  0.703 


Table  6 

Residuals  from  Rotation  Only  Model 


Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

0.487 

0.507 

-0.168 

-0.444 

0.095 

100 

-0.168 

-0.035 

0.098 

-0.151 

-0.064 

120 

-0.035 

-0.380 

0.746 

-0.761 

-0.108 

140 

0.268 

0.162 

-0.145 

-0.251 

0.009 

Average 

0.138 

0.063 

0.133 

-0.402 

-0.017 

Root  Mean  Square  Error  0 . 374 


Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

-1.215 

0.093 

-0.750 

-0.710 

-0.646 

100 

1.183 

0.884 

-0.189 

-0.717 

0.290 

120 

-0.605 

-0.543 

0.485 

0.280 

-0.096 

140 

1.138 

0.384 

-0.055 

0.119 

0.397 

Average 

0.125 

0.205 

-0.127 

-0.257 

-0.014 

Root  Mean  Square  Error 


0.694 


Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

0.150 

-0.089 

0.155 

-0.064 

0.038 

100 

-1.135 

-1.394 

0.240 

0.462 

-0.457 

120 

-0.623 

0.436 

0.159 

1.206 

0.294 

140 

0.113 

-0.359 

-0.177 

0.644 

0.055 

Average 

-0.374 

-0.352 

0.094 

0.562 

-0.017 

Root  Mean  Square  Error  0.623 


Table  7 

Residuals  from  Side  Length  and  Rotation  Model 


Subject  1  (6 

-  -.40) 

Rotation 

Side  Length 

40 

80 

120 

160 

Average 

80 

0.768 

0.788 

0.113 

-0.163 

0.376 

100 

-0.012 

0.122 

0.255 

0.006 

0.093 

120 

-0.127 

-0.471 

0.654 

-0.852 

-0.199 

140 

-0.072 

-0.178 

-0.484 

-0.591 

-0.331 

Average 

0.139 

0.065 

0.135 

-0.400 

-0.015 

Root  Mean  Square  Error 

0.457 

Subject  2  ( 8 

-  -.32) 

Rotation 

Side  Length 

40 

80 

120 

160 

Average 

80 

-0.934 

0.373 

-0.469 

-0.429 

-0.365 

100 

1.339 

1.040 

-0.032 

-0.561 

0.447 

120 

-0.696 

-0.634 

0.393 

0.188 

-0.187 

140 

0.798 

0.044 

-0.395 

-0.221 

0.057 

Average 

0.127 

0.206 

-0.126 

-0.256 

-0.012 

Root  Mean  Square  Error 

0.639 

Subject  3  ( 8  -  -.40) 

Rotation 


Side  Length 

40 

80 

120 

160 

Average 

80 

0.432 

0.192 

0.436 

0.217 

0.319 

100 

-0.978 

-1.237 

0.397 

0.619 

-0.300 

120 

-0.714 

0.345 

0.067 

1.115 

0.203 

140 

-0.227 

-0.699 

-0.517 

0.304 

-0.285 

Average 

-0.372 

-0.350 

0.096 

0.564 

-0.016 

Root  Mean  Square  Error  0.626 


Table  8 

Residuals  From  Two  Strategy  Model 


Subiect  1  (P^ 

00 

1 

&1  -  -.40; 

P2  -  -02, 

*2  -  -• 

39) 

Rotation 

Side  Length 

40 

80 

120 

160 

Average 

80 

0.571 

0.695 

0.124 

-0.049 

0.336 

100 

-0.188 

0.048 

0.285 

0.140 

0.071 

120 

-0.265 

-0.506 

0.723 

-0.680 

-0.182 

140 

-0.171 

-0.174 

-0.377 

-0.380 

-0.276 

Average 

-0.013 

0.016 

0.189 

-0.242 

-0.013 

Root  Mean  Square  Error 

0.405 

Subiect  2  (P^ 

o 

r-~ 

1 

-  --32; 

P2  -  30, 

*2  -  '-: 

51) 

Rotation 

Side  Length 

40 

80 

120 

160 

Average 

80 

-1.011 

0.338 

-0.462 

-0.380 

-0.379 

100 

1.268 

1.011 

-0.019 

-0.505 

0.439 

120 

-0.755 

-0.651 

0.419 

0.256 

-0.183 

140 

0.752 

0.040 

-0.357 

-0.141 

0.074 

Average 

0.064 

0.185 

-0.105 

-0.192 

-0.012 

Root  Mean  Square  Error 

0.629 

Subiect  3  fP 

-  .00, 

0l  -  -.41; 

p2  -  1.00, 

*2  ~  -• 

40) 

Rotation 

Side  Length 

40 

80 

120 

160 

Average 

80 

0.650 

0.300 

0.433 

0.103 

0.371 

100 

■0.786 

-1.156 

0.367 

0.479 

-0.274 

120 

0.575 

0.373 

-0.015 

0.921 

0.176 

140 

0.140 

-0.723 

■0.652 

0.058 

-0.365 

Average 

0.213 

-0.301 

0.033 

0.390 

-0.023 

Root  Mean  Square  Error 

0.577 

Captions  for  Figures 


1.  A  Sample  Task 

2.  Median  Standardized  Log  Response  Time  versus  Rotation 

3.  Median  Standardized  Log  Response  Time  versus  Side  Length 

A.  Posterior  Probabilities  of  Use  of  Rule-Based  Strategy 

5.  Posterior  Probabilities  of  Component  1  Membership  in  a 

Mixture  of  Two  Gaussian  Components 

6.  Posterior  Probabilities  of  Use  of  Rule-Based  Strategy, 
Distinguishing  Instructed  and  Noninstructed  Subjects 
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Median  Standardized  Log  Response  Time 

vs.  Side  Length  —  Rotation  Platted 


Side  Length 


Proportion  of 


of  Component  #1  Membership 


Proportion 


Two  Strategy  Model  —  Trimmed  Data 

Frequency  —  post,  prior  strat  1 

0.45  -| - - 

0.4  - 

0.35  - 


0.3  -4 


____  Probability  of  Using  Strategy  1 

l\  \)  Not  Instructed  Instructed 


0.95 


Distribution  List 


Dr.  Terry  Ackerman 
Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  James  Algina 
1403  Norman  Hall 
University  of  Florida 
Gainesville,  FL  32605 

Dr.  Erling  B.  Andersen 
Department  of  Statistics 
Studiestraede  6 
1455  Copenhagen 
DENMARK 

Dr.  Ronald  Armstrong 
Rutgers  University 
Graduate  School  of  Management 
Newark,  NJ  07102 

Dr.  Eva  L.  Baker 
UCLA  Center  for  the  Study 
of  Evaluation 
145  Moore  Hall 
University  of  California 
Los  Angeles,  CA  90024 

Dr.  Laura  L.  Barnes 
College  of  Education 
University  of  Toledo 
2801  W.  Bancroft  Street 
Toledo,  OH  43606 

Dr.  William  M.  Bart 
University  of  Minnesota 
Dept,  of  Educ.  Psychology 
330  Burton  Hall 
178  Pillsbury  Dr.,  S.E. 
Minneapolis,  MN  55455 


Dr.  Isaac  Bejar 
Mail  Stop:  10-R 
Educational  Testing  Service 
Rosedale  Road 
Princeton,  NJ  08541 

Dr.  Menucha  Birenbaum 
School  of  Education 
Tel  Aviv  University 
Ramat  Aviv  69978 
ISRAEL 

Dr.  Arthur  S.  Blaiwes 
Code  N712 

Naval  Training  Systems  Center 
Orlando,  FL  32813-7100 

Dr.  Bruce  Bloxom 

Defense  Manpower  Data  Center 

99  Pacific  St. 

Suite  155A 

Monterey,  CA  93943-3231. 

Cdt.  Arnold  Bohrer 
Sectie  Psychologisch  Onderzoek 
Rekruterings-En  Selectiecentrum 
Kwartier  Koningen  Astrid 
Bruijnstraat 

1120  Brussels,  BELGIUM 

Dr.  Robert  Breaux 
Code  281 

Naval  Training  Systems  Center 
Orlando,  FL  32826-3224 

Dr.  Robert  Brennan 
American  College  Testing 
Programs 
P.  O.  Box  168 
Iowa  City,  LA  52243 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  John  B.  Carroll 
409  Elliott  Rd.,  North 
Chapel  Hill,  NC  27514 

Dr.  John  M.  Carroll 
IBM  Watson  Research  Center 
User  Interface  Institute 
P.O.  Box  704 

Yorktown  Heights,  NY  10598 

Dr.  Robert  M.  Carroll 
Chief  of  Naval  Operations 
OP-01  B2 

Washington,  DC  20350 

Dr.  Raymond  E.  Christal 
UES  LAMP  Science  Advisor 
AFHRL/MOEL 
Brooks  AFB,  TX  78235 

Mr.  Hua  Hua  Chung 
University  of  Illinois 
Department  of  Statistics 
101  Illini  Hall 
725  South  Wright  St. 
Champaign,  IL  61820 

Dr.  Norman  Cliff 
Department  of  Psychology 
Univ.  of  So.  California 
Los  Angeles,  CA  90089-1061 

Director,  Manpower  Program 
Center  for  Naval  Analyses 
4401  Ford  Avenue 
P.O.  Box  16268 
Alexandria,  VA  22302-0268 


Director, 

Manpower  Support  and 
Readiness  Program 
Center  for  Naval  Analysis 
2000  North  3eauregard  Street 
Alexandria,  VA  22311 

Dr.  Stanley  Collyer 
Office  of  Naval  Technology 
Code  222 

800  N.  Quincy  Street 
Arlington,  VA  22217-5000 

Dr.  Hans  F.  Crombag 
Faculty  of  Law 
University  of  Limburg 
P.O.  Box  616 
Maastricht 

The  NETHERLANDS  6200  MD 

Ms.  Carolyn  R.  Crone 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 

Dr.  Timothy  Davey 
American  College  Testing  Program 
P.O.  Box  168 
Iowa  City,  IA  52243 

Dr.  C.  M.  Dayton 
Department  of  Measurement 
Statistics  &  Evaluation 
College  of  Education 
University  of  Maryland 
College  Park,  MD  20742 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  Ralph  J.  DeAyala 
Measurement,  Statistics, 
and  Evaluation 
Benjamin  Bldg.,  Rm.  4112 
University  of  Maryland 
College  Park,  MD  20742 

Dr.  Lou  DiBello 
CERL 

University  of  Illinois 
103  South  Mathews  Avenue 
Urbana,  IL  61801 

Dr.  Dattprasad  Divgi 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  Hei-Ki  Dong 

Bell  Communications  Research 

6  Corporate  Place 

PYA-1K226 

Piscataway,  NJ  08854 

Dr.  Fritz  Drasgow 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  St. 

Champaign,  IL  61820 

Dr.  Stephen  Dunbar 
224B  Lindquist  Center 
for  Measurement 
University  of  Iowa 
Iowa  City,  LA  52242 

Dr.  James  A.  Earles 

Air  Force  Human  Resources  Lab 

Brooks  AFB,  TX  78235 


Dr.  Susan  Embretson 
University  of  Kansas 
Psychology  Department 
426  Fraser 
Lawrence,  KS  66045 

Dr.  George  Englehard,  Jr. 
Division  of  Educational  Studies 
Emory  University 
210  Fishbume  Bldg. 

Atlanta,  GA  30322 

Dr.  Benjamin  A.  Fairbank 
Operational  Technologies  Corp. 
5825  Callaghan,  Suite  225 
San  Antonio,  TX  78228 

Dr.  P-A.  Federico 
Code  51 
NPRDC 

San  Diego,  CA  92152-6800 

Dr.  Leonard  Feldt 
Lindquist  Center 
for  Measurement 
University  of  Iowa 
Iowa  City,  IA  52242 

Dr.  Richard  L.  Ferguson 
American  College  Testing 
P.O.  Box  168 
Iowa  City,  IA  52243 

Dr.  Gerhard  Fischer 
Liebiggasse  5/3 
A  1010  Vienna 
AUSTRIA 

Dr.  Myron  Fischl 
U.S.  Army  Headquarters 
DAPE-MRR 
The  Pentagon 

Washington,  DC  20310-0300 


Educational  Testing  Service/Mislevy 

Prof.  Donald  Fitzgerald 
University  of  New  England 
Department  of  Psychology 
Armidale,  New  South  Wales  2351 
AUSTRALIA 

Mr.  Paul  Foley 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152-6800 

Dr.  Alfred  R.  Fregly 
AFOSR/NL,  Bldg.  410 
Bolling  AFB,  DC  20332-6448 

Dr.  Robert  D.  Gibbons 
Illinois  State  Psychiatric  Inst. 

Rm  529W 

1601  W.  Taylor  Street 
Chicago,  IL  60612 

Dr.  Janice  Gifford 
University  of  Massachusetts 
School  of  Education 
Amherst,  MA  01003 

Dr.  Drew  Gitomer 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  Robert  Glaser 
Learning  Research 
&  Development  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
Pittsburgh,  PA  15260 

Dr.  Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 


05/24/90 

Michael  Habon 
DORNIER  GMBH 
P.O.  Box  1420 
D-7990  Friedrichshafen  1 
WEST  GERMANY 

Prof.  Edward  Haertel 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

Dr.  Ronald  K.  Hambleton 
University  of  Massachusetts 
Laboratory  of  Psychometric 
and  Evaluative  Research 
Hills'  South,  Room  152 
Amherst,  MA  01003 

Dr.  Delwyn  Hamisch 
University  of  Illinois 
51  Gerty  Drive 
Champaign,  IL  61820 

Dr.  Grant  Henning 
Senior  Research  Scientist 
Division  of  Measurement 
Research  and  Services 
Educational  Testing  Service 
Princeton,  NJ  08541 

Ms.  Rebecca  Hetter 
Navy  Personnel  R&D  Center 
Code  63 

San  Diego,  CA  92152-6800 

Dr.  Thomas  M.  Hirsch 
ACT 

P.  O.  Box  168 
Iowa  City,  LA  52243 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  Paul  W.  Holland 
Educational  Testing  Service,  21-T 
Rosedale  Road 
Princeton,  NJ  08541 

Dr.  Paul  Horst 
677  G  Street,  #184 
Chula  Vista,  CA  92010 

Dr.  Lloyd  Humphreys 
University  of  Illinois 
Department  of  Psychology 
603  East  Daniel  Street 
Champaign,  IL  61820 

Dr.  Steven  Hunka 
3-104  Educ.  N. 

University  of  Alberta 
Edmonton,  Alberta 
CANADA  T6G2G5 

Dr.  Huynh  Huynh 
College  of  Education 
Univ.  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Robert  Jannarone 
Elec,  and  Computer  Eng.  Dept. 
University  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Kumar  Joag-dev 
University  of  Illinois 
Department  of  Statistics 
101  Illini  Hall 
725  South  Wright  Street 
Champaign,  IL  61820 

Dr.  Douglas  H.  Jones 
1280  Woodfem  Court 
Toms  River,  NJ  08753 


Dr.  Brian  Junker 
University  of  Illinois 
Department  of  Statistics 
101  Illini  Hall 
725  South  Wright  St. 

Champaign,  IL  61820 

Dr.  Milton  S.  Katz 
European  Science  Coordination 
Office 

U.S.  Army  Research  Institute 
Box  65 

FPO  New  York  09510-1500 

Prof.  John  A.  Keats 
Department  of  Psychology 
University  of  Newcastle 
N.S.W.  2308 
AUSTRALIA 

Dr.  Jwa-keun  Kim 
Department  of  Psychology 
Middle  Tennessee  State 
University 
P.O.  Box  522 
Murfreesboro,  TN  37132 

Mr.  Soon-Hoon  Kim 
Computer-based  Education 
Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

Dr.  G.  Gage  Kingsbury 

Portland  Public  Schools 

Research  and  Evaluation  Department 

501  North  Dixon  Street 

P.  O.  Box  3107 

Portland,  OR  97209-3107 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  William  Koch 
Box  7246,  Meas.  and  Eval.  Ctr. 
University  of  Texas-Austin 
Austin,  TX  78703 

Dr.  Richard  J.  Koubek 
Department  of  Biomedical 
&  Human  Factors 
139  Engineering  &  Math  Bldg. 
Wright  State  University 
Dayton,  OH  45435 

Dr.  Leonard  Kroeker 
Navy  Personnel  R&D  Center 
Code  62 

San  Diego,  CA  92152-6800 

Dr.  Jerry  Lehnus 

Defense  Manpower  Data  Center 

Suite  400 

1600  Wilson  Blvd 

Rosslyn,  VA  22209 

Dr.  Thomas  Leonard 
University  of  Wisconsin 
Department  of  Statistics 
1210  West  Dayton  Street 
Madison,  WI  53705 

Dr.  Michael  Levine 
Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Charles  Lewis 
Educational  Testing  Service 
Princeton,  NJ  08541-0001 


Mr.  Rodney  Lim 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  St. 
Champaign,  IL  61820 

Dr.  Robert  L.  Linn 
Campus  Box  249 
University  of  Colorado 
Boulder,  CO  80309-0249 

Dr.  Robert  Lockman 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  Frederic  M.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  Richard  Luecht 
ACT 

P.  O.  Box  168 
Iowa  City,  IA  52243 

Dr.  George  B.  Macready 
Department  of  Measurement 
Statistics  &  Evaluation 
College  of  Education 
University  of  Maryland 
College  Park,  MD  20742 

Dr.  Gary  Marco 
Stop  31-E 

Educational  Testing  Service 
Princeton,  NJ  08451 

Dr.  Gessen  J.  Martin 
Office  of  Chief  of  Naval 
Operations  (OP  13  F) 
Navy  Annex,  Room  2832 
Washington,  DC  20350 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  James  R.  McBride 
The  Psychological  Corporation 
1250  Sixth  Avenue 
San  Diego,  CA  92101 

Dr.  Clarence  C.  McCormick 
HQ,  U SMEPCOM/MEPCT 
2500  Green  Bay  Road 
North  Chicago,  IL  60064 

Mr.  Christopher  McCusker 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  St. 

Champaign,  IL  61820 

Dr.  Robert  McKinley 
Educational  Testing  Service 
Princeton,  NJ  08541 

Mr.  Alan  Mead 
c/o  Dr.  Michael  Levine 
Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Timothy  Miller 
ACT 

P.  O.  Box  168 
Iowa  City,  LA  52243 

Dr.  Robert  Mislevy 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  William  Montague 

NPRDC  Code  13 

San  Diego,  CA  92152-6800 


Ms.  Kathleen  Moreno 
Navy  Personnel  R&D  Center 
Code  62 

San  Diego,  CA  92152-6800 

Headquarters  Marine  Corps 
Code  MPI-20 
Washington,  DC  20380 

Dr.  Ratna  Nandakumar 
Educational  Studies 
Willard  Hall,  Room  213E 
University  of  Delaware 
Newark,  DE  19716 

Dr.  Harold  F.  O’Neil,  Jr. 

School  of  Education  -  WPH  801 
Department  of  Educational 
Psychology  &  Technology 
University  of  Southern  California 
Los  Angeles,  CA  90089-0031 

Dr.  James  B.  Olsen 
WICAT  Systems 
1875  South  State  Street 
Orem,  UT  84058 

Dr.  Judith  Orasanu 
Basic  Research  Office 
Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Jesse  Orlansky 
Institute  for  Defense  Analyses 
1801  N.  Beauregard  St. 
Alexandria,  VA  22311 

Dr.  Peter  J.  Pashley 
Educational  Testing  Service 
Rosedale  Road 
Princeton,  NJ  08541 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  Wayne  M.  Patience 
American  Council  on  Education 
GED  Testing  Service,  Suite  20 
One  Dupont  Circle,  NW 
Washington,  DC  20036 

Dr.  James  Paulson 
Department  of  Psychology 
Portland  State  University 
P.O.  Box  751 
Portland,  OR  97207 

Dr.  Mark  D.  Reckase 
ACT 

P.  O.  Box  168 
Iowa  City,  LA  52243 

Dr.  Malcolm  Ree 

AFHRL/MOA 

Brooks  AFB,  TX  78235 

Mr.  Steve  Reiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455-0344 

Dr.  Carl  Ross 
CNET-PDCD 
Building  90 

Great  Lakes  NTC,  IL  60088 
Dr.  J.  Ryan 

Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Fumiko  Samejima 
Department  of  Psychology 
University  of  Tennessee 
31 0B  Austin  Peay  Bldg. 
Knoxville,  TN  37916-0900 


Mr.  Drew  Sands 
NPRDC  Code  62 
San  Diego,  CA  92152-6800 

Lowell  Schoer 

Psychological  &  Quantitative 
Foundations 
College  of  Education 
University  of  Iowa 
Iowa  City,  IA  52242 

Dr.  Mary  Schratz 
905  Orchid  Way 
Carlsbad,  CA  92009 

Dr.  Dan  Segall 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152 

Dr.  Robin  Shealy 
University  of  Illinois 
Department  of  Statistics 
101  Illini  Hall 
725  South  Wright  St. 
Champaign,  IL  61820 

Dr.  Kazuo  Shigemasu 
7-9-24  Kugenuma-Kaigan 
Fujisawa  251 
JAPAN 

Dr.  Richard  E.  Snow 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

Dr.  Richard  C.  Sorensen 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  Judy  Spray 
ACT 

P.O.  Box  168 
Iowa  City,  IA  52243 

Dr.  Martha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  Peter  Stoloff 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  William  Stout 
University  of  Illinois 
Department  of  Statistics 
101  Illini  Hall 
725  South  Wright  St. 
Champaign,  IL  61820 

Dr.  Hariharan  Swaminathan 
Laboratory  of  Psychometric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01003 

Mr.  Brad  Sympson 

Navy  Personnel  R&D  Center 

Code-62 

San  Diego,  CA  92152-6800 

Dr.  John  Tangney 
AFOSR/NL,  Bldg.  410 
Bolling  AFB,  DC  20332-6448 

Dr.  Kikumi  Tatsuoka 
Educational  Testing  Service 
Mail  Stop  03 -T 
Princeton,  NJ  08541 


Dr.  Maurice  Tatsuoka 
Educational  Testing  Service 
Mail  Stop  03-T 
Princeton,  NJ  08541 

Dr.  David  Thissen 
Department  of  Psychology 
University  of  Kansas 
Lawrence,  KS  66044 

Mr.  Thomas  J.  Thomas 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 

Mr.  Gary  Thomasson 
University  of  Illinois 
Educational  Psychology 
Champaign,  IL  61820 

Dr.  Robert  Tsutakawa 
University  of  Missouri 
Department  of  Statistics 
222  Math.  Sciences  Bldg. 
Columbia,  MO  65211 

Dr.  Ledyard  Tucker 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  Street 
Champaign,  IL  61820 

Dr.  David  Vale 
Assessment  Systems  Corp. 
2233  University  Avenue 
Suite  440 

St.  Paul,  MN  55114 

Dr.  Frank  L.  Vidno 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  Howard  Wainer 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  Michael  T.  Waller 
University  of  Wisconsin-Milwaukee 
Educational  Psychology  Department 
Box  413 

Milwaukee,  WI  53201 

Dr.  Ming-Mei  Wang 
Educational  Testing  Service 
Mail  Stop  03-T 
Princeton,  NJ  08541 

Dr.  Thomas  A.  Warm 
FAA  Academy  AAC934D 
P.O.  Box  25082 
Oklahoma  City,  OK  73125 

Dr.  Brian  Waters 
HumRRO 

1100  S.  Washington 
Alexandria,  VA  22314 

Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455-0344 

Dr.  Ronald  A.  Weitzman 
Box  146 

Carmel,  CA  93921 

Major  John  Welsh 
AFHRL/MOAN 
Brooks  AFB,  TX  78223 

Dr.  Douglas  Wetzel 
Code  51 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 


Dr.  Rand  R.  Wilcox 
University  of  Southern 
California 

Department  of  Psychology 
Los  Angeles,  CA  90089-1061 

German  Military  Representative 
ATTN:  Wolfgang  Wildgrube 
Streitkraefteamt 
D-5300  Bonn  2 
4000  Brandywine  Street,  NW 
Washington,  DC  20016 

Dr.  Bruce  Williams 
Department  of  Educational 
Psychology 
University  of  Illinois 
Urbana,  IL  61801 

Dr.  Hilda  Wing 

Federal  Aviation  Administration 
800  Independence  Ave,  SW 
Washington,  DC  20591 

Mr.  John  H.  Wolfe 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 

Dr.  George  Wong 
Biostatistics  Laboratory 
Memorial  Sloan-Kettering 
Cancer  Center 
1275  York  Avenue 
New  York,  NY  10021 

Dr.  Wallace  Wulfeck,  III 
Navy  Personnel  R&D  Center 
Code  51 

San  Diego,  CA  92152-6800 


Educational  Testing  Service/Mislevy 


05/24/90 


Dr.  Kentaro  Yamamoto 
02-T 

Educational  Testing  Sendee 
Rosedale  Road 
Princeton,  NJ  08541 

Dr.  Wendy  Yen 
CTB/McGraw  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 

Dr.  Joseph  L.  Young 
National  Science  Foundation 
Room  320 
1800  G  Street,  N.W. 
Washington,  DC  20550 

Mr.  Anthony  R.  Zara 
National  Council  of  State 
Boards  of  Nursing,  Inc. 
625  North  Michigan  Avenue 
Suite  1544  * 

Chicago,  IL  60611 


